Search AI Products and News

In the field of deep learning, normalization layers are considered an indispensable component of modern neural networks. Recently, research led by Meta FAIR research scientist Zhuang Liu, titled "Transformer without Normalization Layers", has garnered significant attention. This research not only introduces a new technique called Dynamic Tanh (DyT), but also demonstrates the effectiveness of Transformer architectures even without traditional normalization layers.

Although the Transformer model is powerful, its efficiency issue during decoding has always been a headache. However, researchers from the Korea Advanced Institute of Science and Technology, LG, and DeepMind have given us a surprise this time—they have proposed a new Transformer architecture called Block Transformer, which directly boosts decoding speed by 10 to 20 times! But how did they achieve this? It turns out that they "chunked" the attention mechanism of Transformer. This completely over

In today's rapidly advancing field of artificial intelligence, a company named Etched is betting everything on an AI architecture known as Transformer. The company recently announced the launch of the world's first application-specific integrated circuit (ASIC) chip designed specifically for Transformers—Sohu—claiming that its performance far exceeds any GPU currently on the market and will bring revolutionary changes to the AI field.